Lecture 5: Linear Thinking 0.1 Simplest Example: Solving Systems of Linear Equations 0.2 Systems of Linear Inequalities and Linear Programming

According to conventional wisdom, linear thinking describes thought process that is logical or step-by-step (i.e. each step must be completed before the next one is undertaken). Nonlinear thinking, on the other hand, is the opposite of linear: creative, original, capable of leaps of inference, etc. From a complexity-theoretic viewpoint, conventional wisdom turns out to be startlingly right in this case: linear problems are generally computationally easy, and nonlinear problems are generally not. Example: Solving linear systems of equations is easy. Solving quadtratic systems of equations is NP-hard. (Reason: Using the nonlinear constraint x2 = x, we can force variables to be 0/1.) Not all nonlinear problems are difficult, but the ones that turn out to be easy are generally are those that can leverage linear algebra (eigenvalues, singular value decomposition, etc.) In mathematics too linear algebra is simple, and easy to understand. The goal of much of higher mathematics seems to be to reduce study of complicated (nonlinear!) objects to study of linear algebra.